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DETAILED ACTION 

Specification 



1 . The abstract of the disclosure is objected to because it contains a title, "Face 
Detection", and also contains extraneous text at the bottom referring to a figure "Figure 
14". Correction is required. See MPEP § 608.01(b) and 37 CFR 1.72 (b). 

A brief abstract of the technical disclosure in the specification must commence on a separate 
sheet, preferably following the claims, under the heading "Abstract " or "Abstract of the Disclosure. 
" The sheet or sheets presenting the abstract may not include other parts of the application or other 
material. The abstract in an application fded under 35 U.S.C. Ill may not exceed 150 words in 
length. 

The purpose of the abstract is to enable the United States Patent and Trademark Office and the 
public generally to determine quickly from a cursory inspection the nature and gist of the technical 
disclosure. (37 CFR 1.72 (b)) 

The abstract should also.be limited to a single paragraph. Correction is required. 



2. The specification in its current form is without section headings for the 
Background of the Invention, Brief Summary of the Invention, Brief Description of the 
Drawings and the Detailed Description of the Invention. The examiner suggests that the 
applicant update the specification to conform to customary US practice. See MPEP 
§ 608.01 (c-g). 



Claim Rejections - 35 USC § 101 

3. 35 U.S.C. 101 reads as follows: 



Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 
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The USPTO "Interim Guidelines for Examination of Patent Applications for Patent 
Subject Matter Eligibility" (Official Gazette notice of 22 November 2005), Annex IV, 
reads as follows: 

Descriptive material can be characterized as either "functional descriptive material" or "nonfunctional 
descriptive material." In this context, "functional descriptive material" consists of data structures and 
computer programs which impart functionality when employed as a computer component. (The 
definition of "data structure" is "a physical or logical relationship among data elements, designed to 
support specific data manipulation functions." The New IEEE Standard Dictionary of Electrical and 
Electronics Terms 308 (5th ed. 1993).) "Nonfunctional descriptive material" includes but is not limited 
to music, literary works and a compilation or mere arrangement of data. 

When functional descriptive material is recorded on some computer-readable medium it becomes 
structurally and functionally interrelated to the medium and will be statutory in most cases since use of 
technology permits the function of the descriptive material to be realized. Compare In re Lowry, 32 
F.3d 1579, 1583-84, 32 USPQ2d 1031, 1035 (Fed. Cir. 1994) (claim to data structure stored on a 
computer readable medium that increases computer efficiency held statutory) and Warmerdam, 33 
F.3d at 1 360-61 , 31 USPQ2d at 1 759 (claim to computer having a specific data structure stored in 
memory held statutory product-by-process claim) with Warmerdam, 33 F.3d at 1 361 , 31 USPQ2d at 
1760 (claim to a data structure per se held nonstatutory). 

In contrast, a claimed computer-readable medium encoded with a computer program is a computer 
element which defines structural and functional interrelationships between the computer program and 
the rest of the computer which permit the computer program's functionality to be realized, and is thus 
statutory. See Lowry, 32 F.3d at 1583-84, 32 USPQ2d at 1035. 



Claims 18-20 are rejected under 35 U.S.C. 101 because the claimed invention is 
directed to non-statutory subject matter as follows. Claim 18 defines computer software 
embodying functional descriptive material. Claim 19 defines a medium providing 
program code embodying functional descriptive material. Claim 20 defines a storage 
medium embodying functional descriptive material. However, the claims do not define a 
computer-readable medium or computer-readable memory and are thus non-statutory 
for that reason (i.e., "When functional descriptive material is recorded on some 
computer-readable medium it becomes structurally and functionally interrelated to the 
medium and will be statutory in most cases since use of technology permits the function 
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of the descriptive material to be realized" - Guidelines Annex IV). The scope of the 
presently claimed invention encompasses products that are not necessarily computer 
readable, and thus NOT able to impart any functionality of the recited program. The 
examiner suggests amending the claim(s) to embody the program on "computer- 
readable medium" or equivalent; assuming the specification does NOT define the 
computer readable medium as a "signal", "carrier wave", or "transmission medium" 
which are deemed non-statutory (refer to "note" below). Any amendment to the claim 
should be commensurate with its corresponding disclosure. 

Note: 

"A transitory, propagating signal ... is not a "process, machine, manufacture, or 
composition of matter." Those four categories define the explicit scope and reach of 
subject matter patentable under 35 U.S.C. § 101; thus, such a signal cannot be 
patentable subject matter." (In re Petrus A.C.M. Nuijten; Fed Cir, 2006-1371 , 
9/20/2007). 

Should the full scope of the claim as properly read in light of the disclosure 
encompass non-statutory subject matter such as a "signal", the claim as a whole would 
be non-statutory. In the case where the specification defines the computer readable 
medium or memory as statutory tangible products such as a hard drive, ROM, RAM, 
etc, as well as a non-statutory entity such as a "signal", "carrier wave", or "transmission 
medium", the examiner suggests amending the claim to include the disclosed tangible 
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computer readable media, while at the same time excluding the intangible media such 
as signals, carrier waves, etc. 

Claim Rejections - 35 USC § 101 

35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

Claim 21 is rejected under 35 U.S.C. 101 because the claimed invention is 
directed to non-statutory subject matter as follows. Claim 21 as a whole defines a 
transmission medium and "[a] transitory, propagating signal ... is not a "process, 
machine, manufacture, or composition of matter." Those four categories define the 
explicit scope and reach of subject matter patentable under 35 U.S.C. § 101; thus, such 
a signal cannot be patentable subject matter." (In re Petrus A.C.M. Nuijten; Fed Cir, 
2006-1371, 9/20/2007). 



Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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5. Claims 1, 2, 4-6, 15, 17 and 18 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Rui et al. (US 2003/01 03647). 

Regarding claims 1 and 17, Rui discloses [a] face detection apparatus ("FIG. 17 
illustrates a general computer environment 600, which can be used to implement the 
automatic detection and tracking of multiple individuals described herein", Rui, 
paragraph 210 and figure 17) for tracking a detected face between images in a video 
sequence, the apparatus comprising: 

a first face detector for detecting the presence of face(s) in the images (Rui, 
Figure 7, reference 324); a second face detector for detecting the presence of face(s) in 
the images (Rui, figure 7, reference 334); the first face detector having a higher 
detection threshold than the second face detector, so that the second face detector is 
more likely to detect a face in an region in which the first face detector has not detected 
a face (Rui, figure 7, reference 334 Multi-view face detection and "Multi-view face 
detection module 166 uses one or more detection processes that attempt to detect 
human faces in different poses or from multiple views (that is, to detect the faces even 
though the head may be tilted, rotated away from the image capturing device, etc.). Any 
of a wide variety of face detection techniques can be used by multi-view face detection 
module 166", Rui, paragraph 92, the multi-view technique is more likely to find a match 
in that it is designed to match against multiple orientations rather than the single 
orientation used by the first detector); and 
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a face position predictor for predicting a face position in a next image in a test 
order of the video sequence on the basis of a detected face position in one or more 
previous images in the test order of the video sequence; in which: if the first face 
detector detects a face within a predetermined threshold image distance of the 
predicted face position, the face position predictor uses the detected position to produce 
a next position prediction ("a prediction is made as to where the object will be in the 
current frame t based on the tracking results from the previous frame t-1 and the 
object's dynamics", Rui, paragraph 108); 

if the first face detector fails to detect a face within a predetermined threshold 
image distance of the predicted face position, the face position predictor uses a face 
position detected by the second face detector to produce a next position prediction (Rui, 
figure 7, the second detector only operates if the first detector is unsuccessful, so if the 
second detector detects a face, the position information from the second detector will be 
used to calculate the next position). 

Regarding claim 2, Rui teaches [apparatus according to claim 1 , in which the 
first face detector is operable: 

to derive a set of attributes from regions of each successive image ("During 
operation, detection and tracking module 132 analyzes content on a frame by frame 
basis", Rui, paragraph 41 ); 
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to compare the derived attributes with attributes indicative of the presence of a 
face ("If the region does resemble a human upper body, then the portion of the region 
including the head (but excluding the shoulders) is extracted", Rui, paragraph 69); 

to derive a probability of the presence of a face by a similarity between the 
derived attributes and the attributes indicative of the presence of a face ("hierarchical 
verification module 142 generates a new confidence level for the region", Rui, 
paragraph 43); and 

to compare the probability with a threshold probability ("When re-verifying a 
region, hierarchical verification module 142 generates a new confidence level for the 
region and compares the confidence level to the threshold value", Rui, paragraph 43) 

Regarding claim 4, Rui teaches an [apparatus according to claim 1 , in which the 
second face detector is operable to compare the colours of image regions with colours 
associated with human skin ("Multi-view face detection module 166 uses one or more 
detection processes that attempt to detect human faces in different poses or from 
multiple views", Rui, paragraph 92, Any method of analysis used by Rui's second 
detector will need to make use of pixel intensity or color since that is really the only 
information that an individual pixel carries). 



Regarding claim 5, Rui teaches an [apparatus according to claim 4, the 
apparatus being operable to discard a face track if the second detector detects that the 
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detected face differs by more than a threshold amount from a skin colour (Rui, figure 7, 
if the second face detector 334, does not find a face 336, the face track is dropped 338). 

Regarding claim 6, Rui teaches [apparatus according to claim 1 , in which the 
face position predictor is initiated only in response to a face detection by the first face 
detector ("the position of the face can be estimated at a coarse level based on the 
located set of points, and then this coarse estimation used as the initial guess in the 
parametric contour -tracking process discussed above", Rui, paragraph 162). 

Regarding claim 15, Rui teaches [vjideo conferencing apparatus comprising 
apparatus according to claim 1 ("FIGS. 1 and 2 illustrate exemplary environments in 
which robust automated detection and tracking can be used. In FIG. 1, multiple (n) 
video conferencing systems 102 are able to communicate audio/video content to one or 
more of each other", Rui, paragraph 27 and figure 1) 

Regarding claim 18, Rui teaches [c]omputer software having program code for 
carrying out a method according to claim 17 ("FIG. 7 is a flowchart illustrating an 
exemplary process 320 for performing hierarchical verification. Process 320 is 
performed by hierarchical verification module 142 of FIG. 3, and may be performed in 
software", Rui, paragraph 84). 
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6. Claim 3 is rejected under 35 U.S.C. 103(a) as being unpatentable over Rui et al. 
(US 2003/0103647) in combination with Moghaddam et al. (IEEE, TP AMI Vol 19, 1997). 

Regarding claim 3, Rui teaches [apparatus according to claim 2. 
Rui does not teach in which the attributes comprise the projections of image 
areas onto one or more image eigenvectors. 

Moghaddam, working in a similar problem solving area of face recognition does 
teach attributes that comprise the projections of image areas onto one or more image 
eigenvectors ("we must incorporate the underlying probability distribution of the object. 
Subspace methods and eigenspace decompositions are particularly well-suited to such 
a task", Moghaddam, page, 696, Section 1.1, first paragraph). 

It would have been obvious at the time the invention was made for one of 
ordinary skill in the art to use the eigen-space method as taught by Moghaddam in the 
face detection and tracking system of Rui to implement a face recognizer that can 
provide a compact and parametric description of the object's appearance and also 
automatically identify the degrees-of-freedom of the underlying statistical variability 
(Hoghaddam, page 697, first paragraph). It also would allow making generic detection 
of faces possible as opposed to more stringent template matching methods. 
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7. Claims 7 and 10 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Rui et al. (US 2003/0103647) in combination with Zhang et al. (US 6, 298,145). 

Regarding claim 7, Rui teaches an [apparatus according to claim 1. 

Rui does not teach if the face detectors fail to detect a face within a 
predetermined threshold image distance of the predicted face position, the face position 
predictor uses the predicted face position to produce a next position prediction. 

Zhang, working in the same problem solving are of face detection does teach if 
the face detectors fail to detect a face within a predetermined threshold image distance 
of the predicted face position, the face position predictor uses the predicted face 
position to produce a next position prediction ("even if the face detector 41 fails to detect 
any face in the current I frame, the face tracking information from the face tracker 42 
may still cause the face detector 41 to regard the current I frame as a face image 
frame", Zhang, column 9, line 3). 

It would have been obvious at the time the invention was made to use the 
prediction extension method of Zhang with the face tracking system of Rui to attempt to 
preserve tracking when a noise or a temporary obstruction defeats the face detector for 
a given frame. 
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Regarding claim 10, the combination of Rui and Zhang teaches an [apparatus 
according to claim 1 , in which if two faces are being tracked in respect of an image, one 
track is discarded so that: 

a track based on a detection by the first detector has priority over a track based 
on a detection by the second detector or a predicted position (Rui, figure 7, the second 
detector only operates if the first detector is unsuccessful, so if the first detector finds a 
face, it has priority over the second ); and 

a track based on a detection by the second detector has priority over a track 
based on a predicted position ("even if the face detector 41 fails to detect any face in 
the current I frame, the face tracking information from the face tracker 42 may still cause 
the face detector 41 to regard the current I frame as a face image frame", Zhang, 
column 9, line 3, Zhang only extends the previous prediction when the detector fails, 
similar to the above condition, the second detector has priority over the tack based on a 
previous predicted position as a successful detection prevents extending the previous 
position prediction). 

8. Claim 14 is rejected under 35 U.S.C. 103(a) as being unpatentable over Rui et al. 
(US 2003/0103647) in combination with Magarey (US 2003/0053661). 

Regarding claim 14, Rui teaches an [apparatus according to claim 1. 
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Rui does not teach an apparatus being operable to discard a face track if the 
detected face has an inter-pixel variance lower than a first threshold amount or higher 
than a second threshold amount. 

Magarey, working in the same problem solving area of feature tracking does 
teach to discard a face track if the detected face has an inter-pixel variance higher than 
a threshold amount ("comparing said feature data with reference data, wherein a 
difference between said feature data and said reference data that is larger than a first 
predetermined number indicates that track of said feature has been lost", Magarey, 
paragraph 5). 

It would have been obvious at the time the invention was made for one of 
ordinary skill in the art to use the thresholded distance test of Magarey with the face 
detection and tracking system of Rui to determine when a loss of tracking has occurred 
to avoid useless computation when the target has disappeared ('The feature may be 
occluded by another object, or suddenly change course, so that predictive motion 
estimation fails and tracking is lost. In these cases, tracking should be halted and the 
system notified of the "loss-of -track" (LOT) condition. However, the nature of motion 
estimation is such that a vector is always returned whether or not the feature is still 
actually visible near the predicted position", Magarey, paragraph 8). 
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9. Claim 16 is rejected under 35 U.S.C. 103(a) as being unpatentable over Rui et al. 
(US 2003/0103647) in combination with Li (US 2002/0150280). 

Regarding claim 16, Rui teaches an apparatus according to claim 1. 

Rui does not teach using his apparatus to implement a [surveillance apparatus. 

Li working in the same problem solving area of face detection does teach the use 
of face detection in surveillance ("face detection also has numerous other applications 
including surveillance systems", Li, paragraph 4). 

it would have been obvious at the time the invention was made for one of 
ordinary skill in the art to apply the face detection and tracking method of Rui in a 
surveillance system as suggested by Li in order to find other applications for his 
apparatus and to increase marketing potential. 

Allowable Subject Matter 

10. Claims 8, 9 and 1 1-13 are objected to as being dependent upon a rejected base 
claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 
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1 1 . The following is a statement of reasons for the indication of allowable subject 
matter: 

Regarding claim 8, the prior art of record does not teach discarding a face 
tracking position if for more than a specified proportion of images, a predicted position is 
used to make further position predictions. 

Regarding claim 9, the prior art of record does not teach discarding a face 
tracking detection if a secondary detector is relied upon for position prediction for more 
than a predetermined proportion of images. 

Regarding claim 1 1 , the prior art of record does not teach if a detector is tracking 
two faces, the track corresponding to the smaller face is discarded. 

Regarding claim 12, the prior art of record does not teach at least two 
consecutive detections by a first detector is required before starting a face track. 

Regarding claim 1 3, the prior art of record does not teach where at least g face 
detections are required every n frames in order to maintain a face track. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Thomas M. Redding whose telephone number is (571) 
270-1579. The examiner can normally be reached on Mon - Fri 7:30 am - 5:00 pm EST. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Vikkram Bali can be reached on (571 ) 272-7415. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. A 
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